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ABSTRACT 


Repeatable accuracy of hydrographic positioning was 
examined in terms of the two-dimensional normal distribution 
function which results in an elliptical error figure. The 
error ellipse was discussed, and two methods for conversion 
of elliptical errors to circular errors were given. These 
methods are "circle of equivalent probability" and "root 
mean square error" . Using the d^^^^ error concept, 

repeatable accuracy of ranging, azimuthal, and hyperbolic 
systems was evaluated, and methods were developed to draw 
repeatability contours for those systems. 

A brief theoretical background was provided to explain 
the method of least squares and discuss its application to 
hydrographic survey positioning. For ranging, hyperbolic, 
azimuthal, sextant angle, and Global Positioning System the 
least squares observation equations were developed. Specific 
examples were constructed to demonstrate the capabilities 
of this data adjustment technique when applied to redundant 
position observations. 
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I. INTRODUCTION 


Positioning of the survey vessel is equal in importance 
with depth determination in the collection of hydrographic 
survey data. Fundamental to an understanding of the accuracy 
of position information is an analysis of the various errors 
and their sources which must be either eliminated, compen¬ 
sated for, or otherwise modeled. The result of this analysis 
is that the reliability of position data can be evaluated 
and used to estimate the overall accuracy of hydrographic 
soundings. 

Once these potential error sources are understood, 
methods must be developed to quantify accuracy. Much 
research has been conducted in this area in the past. One 
purpose of this thesis is to collect and present useful 
concepts of error theory which apply directly to hydrographic 
survey. Simple graphical techniques were developed which 
can be used to produce accuracy contours as a function of 
the survey net geometry. 

Conventional survey techniques rely on only two lines 
of position (LOP) to determine a positioning fix. This 
introduces the possibility of significant error. 

In navigation, although inherently less accurate than 
positioning due to the techniques and systems used to 
determine the LOP's, three LOP*s are required to produce a 
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fix. Position is adjusted graphically by placing the fix 
in the center of the triangle formed by the three inter¬ 
secting LOP'S. This concept of taking one redundant 
observation can lead to significant improvement in hydro- 
graphic survey positioning data. Mathematical adjustment 
techniques such as the method of least squares may be used 
to determine the best estimate of position. 

Least square adjustments are commonly performed on land 
survey data where redundant observations are easily made. 
With the advent of new positioning systems and computer 
technology, making redundant observations at sea is no 
longer impractical. The second purpose of this thesis 
is to explain the basic method of least squares, and to 
formulate examples of the least squares adjustment pro¬ 
cedure applied to specific types of hydrographic survey 
systems. This data adjustment technique not only provides 
the best estimate of position but also may be used to deter¬ 
mine the absolute positioning accuracy associated with each 
data point in a hydrographic survey. 
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II. REPEATABLE ACCURACY OF HYDROGRAPHIC 
SURVEY POSITIONS 


A. TYPES OF ERRORS 

It is impossible to make measurements o£ physical data 
without making errors. These measurement errors may be 
classified in the following manner. 

1. Blunders 

These are mistakes which result from misreading 
instruments, transposing figures, faulty computations, etc. 

They may be large and easily observed, or smaller and less 
detectable, or very small and indistinguishable in the 
data. Blunders are usually detected through comparing repeated 
measurements, careful editing, and procedural checks in the 
data collection process. Physical measurements will contain 
a constant bias if these errors are not removed from the data 
set. 

2. Systematic Errors 

Uncalibrated instruments or environmental factors, 
such as temperature and humidity changes which affect the 
performance of the measuring instruments, will induce system¬ 
atic errors into the observations. The occurrence of this 
type o£ error may result in a pattern which can be recognized 
and mathematically modeled. The simplest pattern to model 
would be some observable trend in the data of constant 
magnitude and direction. Such a trend can easily be 
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subtracted from the observations to remove the systematic 
error. 

If numerous systematic errors exist, or the errors 
are such that they cannot be accurately modeled, then their 
effect on the data must be estimated by calibration. Cali¬ 
bration is the process of comparing the measuring instrument 
against a known standard. The difference between the observed 
and known value may be used as an estimate of the total effect 
of all systematic errors present. Thus, calibration provides 
a "corrector" which must be applied to the data set. Examples 
of important systematic errors in hydrographic survey position¬ 
ing include instrument errors, errors in positioning control 
points, and variations in the propagation velocity of electro¬ 
magnetic energy. 

3. Random Errors 

These errors result from accidental and unknown 
causes. Their effect cannot be removed from the observations 
and, therefore, must be quantified statistically. Random 
errors have certain characteristics which facilitate such an 
approach. Positive and negative errors occur with equal 
frequency, small errors are more probable than large errors, 
and extremely large errors rarely occur. 

The frequency distribution of random errors can be 
modeled mathematically by the normal distribution function. 
Assuming all measurement errors are independent and random. 
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thereby conforming to the normal distribution, measurement 
accuracy can be specified statistically by defining a con¬ 
fidence interval around the best estimate of the measured 
value. Procedures for computing these intervals are reviewed 
in Appendix A. 

B. .ACCURACY OF HYDROGRAPHIC POSITIONS 

The achievable accuracy of a hydrographic survey 
positioning system is best described by defining the follow¬ 
ing terms: repeatability and predictability. 

Repeatability is a measure of the accuracy with which 
the positioning system permits the user to return to a 
specific point on the surface of the earth defined in terms 
of the lines of position generated by the system. Included 
in repeatability are the effects of random errors, errors 
due to net geometry, and errors resulting from the angle 
of intersection for the two lines of position that establish 
a fix. Repeatable accuracy is therefore a measure of the 
relative accuracy of a positioning system. Unresolved biases 
exist in hydrographic positions due to the presence of 
systematic errors that have not been subtracted from the 
data or compensated for as a result of calibration. 

Predictability is the measure of accuracy with which the 
system can define the location of the same point in terms 
of geographic (or geodetic) coordinates rather than simply 
the intersection of two lines of position. Thus, predictable 
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accuracy is an absolute accuracy. Using conventional hydro- 
graphic survey techniques, predictability could be achieved 
only if all systematic errors were removed from the data so 
that only the effects of random errors, net geometry, and 
intersection angle remain. For example, the lattice generated 
by an electronic positioning system is distorted primarily as 
a result of the variability in the propagation velocity of 
electromagnetic energy. Ideally, if there was no distortion 
of the electronic lattice, then the accuracy of a position, 
corrected for any remaining systematic errors, could be 
quantified statistically in terms of predictable accuracy. 
However, since these distortions exist, the effective velocity 
of propagation would have to be accurately modeled through¬ 
out the survey area. Then it would be possible to subtract 
the effects of this systematic error and derive positions 
in terms of predictable geographic coordinates. Research 
is currently being conducted to quantify the parameters which 
affect propagation velocity in order to model these values 
for such application [Ref. 19]. 

A second method to achieve predictable accuracy is by 
making redundant observations to establish hydrographic 
survey positions. If three intersecting lines of position 
are available instead of the usual two, the resulting fix 
is overdetermined, and data adjustment techniques must be 
applied. The method of least squares is most useful in 
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adjusting such data. Through the application of least 
squares adjustment techniques, the best estimate of position 
is found and the position's predictable accuracy is resolved. 

A complete discussion of this procedure is presented in 
Section III. 

C. REPEATABLE ACCURACY 

In the determination of hydrographic positions, blunders 
are eliminated by observing strict survey procedures, and 
system calibration is performed in an attempt to remove system¬ 
atic errors. Because some systematic errors still remain, 
the accuracy of hydrographic positions must be stated in 
terms of repeatability. 

The modeling of random errors is done by using the two- 
dimensional normal distribution function. When the normal 
distribution is applied to the positional errors, the result¬ 
ing error figure is an ellipse. 

1. Elliptical Errors 

Hydrographic positions are determined by the inter¬ 
section of two lines of position (LOP). Because of the errors 
in each LOP, the actual position may lie somewhere between 
the error limits (shown as additional arcs either side of 
LOP’S in Figure II-1). 

The intersection of the two LOP's, together with the 
standard errors associated with each, is drawn to an expanded 
scale in Figure II-2. By applying the two-dimensional normal 
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Figure II-l: Position location at the intersection of 
two lines of position. 
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Figure II-2: Expanded view of the intersection of 
two lines of position and the associated error ellipse. 














































distribution to positional errors, it is seen that the contours 
of equal probability density about such an intersection are 
ellipses with their center at the intersection point. 

For simplicity in the discussion, the following assump¬ 
tions are made: 

1. Only errors contributing to repeatable accuracy are 
considered. 

2. The random errors associated with each LOP are 
assumed to be normally distributed. 

3. The random errors in each LOP are assigned to be 
independent, i.e., a change in the error of one 
LOP has no effect upon the other. 

4. The LOP'S are assumed to be straight lines in the 
small area in the immediate vicinity of their 
intersection. 

5. Errors of position are limited to the two- 
dimensional case. 

As shown in Figure II-2, the general case of the inter¬ 
section of two LOP'S at any angle and with different values 
of errors associated with each LOP results in an elliptical 
error figure. 

It is readily seen from Figure II-2 that the exact shape 
of the error ellipse varies with the magnitudes of both of 
the one-dimensional LOP errors, and a^, as well as with 
the angle of intersection, 3. 

The values of the semi-major and serai-minor axes of the 
error ellipse (using one a error) are given by the following 
equations [Refs. 4 and 5]. 
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Semi-major axis : 


= —i—r c ) + \/(tr,Vo-^)-(^srK^p) (?,^(5:a ' 1 • 


(Il-la) 


Semi-minor axis: 


(5-y^=J: 


ZSin'^P 


+ 1 (II.lb) 


Generally - <^2 ~ then equations (Il-la) and (Il-lb) 
simplify to 


ay = 


^S\n (^/2) 


and 




^Cos(^/2) . 


CII-2) 


After computing the semi-major and semi-minor axes, the 
probability of the error ellipse is given by the distribution 
function 

iL 

P(y.,y) = i - e'^ , CH-3) 


where 
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The solution of equation II-3 with values of h for different 


probabilities yields the results shown in Table II-l. 


Probability (%) 

h 

39,35 

1.0000 

50.00 

1.1774 

63.21 

1.4142 

90,00 

2.1460 

99.00 

3.0349 

99.78 

3.5000 


Table II-l: Values of constant h. 


For example, for 39.35% probability the axes of the 

ellipse are 1.00 a and 1.00 a ; for 50% probability the axes 

X y 

are 1.1774 o and 1.1774 a . Figure II-3 shows the error 

X y 

ellipses for different values of h. 

The angle, 9, between the semi-major axis of the error 
ellipse and the line of position which has smaller standard 
error is given by [Ref. 5] 


4*00 2 . © 


Sin 1 ^ 


CII-4) 







h=4142 
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Figure II-3: Contours of equal probability areas. 









































where is the smaller standard error. In the case of 
^1 ~ *^2’ II-4 simplifies to 

e =-&- . (II-5) 

2 

The importance of the angle 0 is that it specifies the 
orientation of the error ellipse according to the lines of 
position. 

2. Circular Error Approximations 

In general, the use of the error ellipse is compli¬ 
cated by the problem of axis orientation and the propagation 
of elliptical errors. Therefore, in order to simplify 
probability calculations and avoid the above problems, the 
elliptical errors are approximated by circular errors which 
are easier to use and understand. The accuracy of a hydro- 
graphic position may then be stated in terms of a circle of 
specified radius about the point. 

Note that when the angle of intersection is a right 
angle and the two errors are equal, the error ellipse becomes 
a circle and is described by the circular normal distribution. 
Generally, this is not the case, and elliptical errors must 
be converted to circular errors. This is done by using either 
the circle of equivalent probability or the root-mean-square 
error concept. 

a. Circle of Equivalent Probability 

A circle of equivalent probability is obtained 
utilizing an existing table for the two-dimensional normal 
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distribution (Table II-2). This table is used with the 
two standard errors along the semi-major and semi-minor axes 
of the error ellipse (Equations Il-la, Il-lb or II-2). To 
find the radius of equivalent probability, equations II-la, 
Il-lb or II-2 must first be utilized to obtain the values of 


a and a,, 
X y 


To enter the table the following ratios are needed: 


c = — where o is the greater standard error 
a X ° 


and 


y. ^ Radius of circle of equivalent probability 

Greater standard error > 

where K is the conversion factor needed to solve for the 
radius (R) of the circle of equivalent probability. 

The table relates varying values of ellipticity 
to the radius of circles of equivalent probability. Enter 
the table with the computed values for c and K to determine 
the probability for a circle of given radius, or alternately, 
for a given value of probability, determine the radius of the 
error circle. 

EXAMPLE II-l: The two standard errors of a positioning 
system estimated from field observations are = o^ ~ 6 meters. 

To determine the probability of location within a circle of 
10 m radius when the angle of intersection, 3, is 60°, 
equation II-2 must be used to find a and a : 

X y 
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Table II-2: Circular error probabilities 
CBovrditch, 1977). 
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^_ =. 8-^6m 

iz 5in Wl 'TT SCo 


(5*0 = ^ ^ - = 4.9 m . 

>Ii Cos p/2. Cos ^ 

2 . 

Using the ratio, c = — .Jtl?— =. . 5 B and 

fl-% 8-4B 

C rotiios Q-f c-loc-Le. _ tO ^ i«2. , 

<5‘k 8.^8 

enter Table II-2 with K = 1.2 and c = .58 - .6. The proba¬ 
bility is found to be approximately 67%. (The value in the 
table is .6714269.) 

EXAMPLE II-2: For the system described in example II-l, 
the radius of the error circle with 90% probability may be 
determined. 

First, entering Table II-2 with c = .6, for 90 
probability (the closest table value is .9019110), K is 
found to be 1.8. The radius of the error circle is equal 
to K times a : 

X 

1.8 X 8.48 = 15.3 meters. 
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Table II-3 is more convenient for solving 
problems such as in example II-2 because the table is entered 
by using values of c and probability, P, in order to solve 
for the conversion factor, K. Note that the error circles 
identifying the 501 probability area (circular error 
probable, or CEP) and 90% area (circular map accuracy 
standard, or CMAS) are the most frequently used probability 
intervals. 

For constant values of and a^, circular error 

probabilities vary as a function of the angle of intersection, 

3, of the lines of position. To simplify the investigation of 

geometrical effects, the common case of = 02 = cr will be 

considered. Under this condition, the equations for a and o 

X y 

simplify to equation II-2. Taking the ratio of these two 
values, c is found to be C =. =■ . Using 

the simplified equations, significant parameters of the error 
ellipse have been listed in Table II-4 as a function of the 
intersection angle, 3, for the 50% probability interval (CEP) 
and in Table II-5, for the 90% probability interval. The 
data shows that the radius of the error circle, R, increases as 
the angle of intersection decreases. In the last columns, 
the error factor is defined as 


Error factor 


R (at any intersection angle) 

K'(at'3 = 90"^)- 
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or a multiplier by which the error circle radius, R, at any 
intersection angle may be computed from the radius of the error 
circle at 3 = 90°. For example, from Table II-4 it is seen 
that at a 50° intersection angle, R is 1.206 times greater 
than the radius at 3 = 90°. 

As seen in Tables II-4 and II-5, the optimum 
accuracy is obtained when the intersection angle, 3, is 90°. 

It can be said that the geometric dilution of precision (GDOP) 
is minimum for a 90° intersection angle. Thus, the error 
factor defined in Tables II-4 and II-5 is commonly known 
as GDOP. Effects of geometric dilution are shown in Figure 
II-4 for CEP and 901 probability interval (CMAS). Acceptable 
intersection angles for LOP's used in fixing hydrographic 
positions usually range between the limits of 30° and 150°. 

As seen in Figure II-4, the radius of the 90° probability 
interval circle is increased by a factor of two near the 
acceptable limits for hydrographic fix angles. Correspond¬ 
ingly, positioning accuracy is decreased by a factor of two. 

b. Root-Mean Square Error (d_ ) 

^ ^ rms 

The root-mean-square error, d , is defined as 

the square root of the sum of the squares of the error 

components along the major and minor axes of the error ellipse. 

To calculate the d error, first equations II-la, lb or 

rms ’ 

II-2 are utilized to obtain the values of a and a . Then 

X y 

the definition of d,„„^ is used, 

rms ’ 
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Figure II-4; Geometric dilution 

of precision for CEP and 90% probability 
interval (Bowditch, 1977). 


= i/^T^ . ( 11 - 6 ) 


where a = o is the semi-major axis of the error ellipse 
and b = CTy. is the semi-minor axis. 

Alternately, formulas II-la and II-lb are sub¬ 
stituted into the definition of d_ error (equation II-6) 

rius 

and a more useful form of d__ is obtained in terms of 

rms 

O 2 and the angle of intersection, B: 


6 - ^ 


\|cr,^ + 






(11-7) 


Figure II-5 illustrates the definition of d error. 

rms 
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Figure II-5: Illustration of root mean square error. 

One d is defined as the radius of the error circle 
rms 

obtained using one and one Oy as the semi-major and semi¬ 
minor axes of the error ellipse. Two is defined as the 

radius of the error circle obtained using two times the 

a and a„ values. 

X y 

The value of does not correspond to a fixed 

probability interval for given values of and It 

corresponds to a fixed probability interval only when 
0 = 90® and o, =» o- so that the resulting probability figure 
is a circle. In the elliptical cases, the probability 
associated with a fixed value of d^jj^g varies as a function of 
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the eccentricity of the error ellipse. This can easily be 
seen with an example using Table II-2. 

First, consider = 15 m, = 10 m, 

drms = = 18 m , 

C=®k.=.fe66 ond K = ^ Ji- =1.^ . 

cv Ctt \5 

For c = .666 table values must be interpolated. Enter 
Table II-2 with c = .6 and c = .7 for K = 1.2. The correspond¬ 
ing table values are found to be .6714269 and .6306168. 

Thus the probability of 18 m d is found to be 64.78%. 

^ ' rms 

Secondly, consider a = 17 m, a = 6 m 

X y 

= \l = 18 m , 

c = VlT = • 353 onci •= _ i.o59 . 

IT- 

The interpolated probability from Table II-2 is 67.4%. 

As seen above for the two cases, d errors are equal but 

c values (eccentricity) are different. As a result the 

corresponding probabilities are 64.78% and 67.4%. 

Table II-6 shows the variations in probability 

associated with the values of 1 d_„„'and 2d as a 

rms rms 

function of eccentricity (o /a ), and Figure II-6 shows the 

y ^ 

same information graphically for 1 d error. 

® ^ ' rms 
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Probability 


1 d 


rms 


2 d 


rms 


0 

.683 

.954 

.1 

.682 

.955 

.2 

.682 

.957 

.3 

.676 

.961 

.4 

.671 

.966 

.5 

.662 

.969 

.6 

.650 

.973 

.7 

.641 

.977 

.8 

.635 

.980 

.9 

.632 

.981 

1.0 

.632 

.982 


Table II-6: Variations in probability as a function of 
eccentricity (Bowditch, 1977). 



Figure II-6: Variation in with ellipticity (1 

CBowditch, 1977). 
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As seen in Table II-6, the probability that the 
position will be within the 1 error circle ranges from 

68.3% when a = 0.0 to 63.2% when o = a and two d 

V V X illO 

ranges from 95.4% to 98.2%, respectively. 

In Equation II-7, the error was given 

assuming the errors in each line of position are independent. 
If the measurement of line of position #1 is related to 
measurement of line of position #2, then there is correla¬ 
tion between o^^ and o^; e.g., is dependent on ^ 

change in produces a corresponding change in 02 * In 
this case, the equation for the root mean square position 
error is given as 


^rrrvS-- ^‘ 2 . 2.p cr-iOoS & 

Svn^ V Ui- 

where ^ is the correlation coefficient between and 02 . 

Two different derivations of this equation are presented in 
the following papers: Bigelow (1963) and Heinzen (1977) 

[Refs. 2 and 10]. 

In summary, root mean, square error is easy to 
obtain mathematically, and it yields relative values of 
accuracy which are normally understood. Therefore, in subse¬ 
quent sections, d^^^^ will be used to explain the repeatable 
accuracies of hydrographic positioning systems. 
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D. REPEATABLE ACCURACY OF HYDROGRAPHIC POSITIONING SYSTEMS 


1. R anging Systems 

In ranging systems, the lines of positions are drawn 
as circles centered about each control station. The repeat¬ 
ability of this type of system is a function of the inter¬ 
section angle, 3, and the random errors associated with each 
line of position. 

The two ranges are independent of each other. There¬ 
fore, the correlation coefficient, p, is zero, and d is 
given by Equation II-7 which is repeated here : 

'*rms ' . (III-9) 

S\np 

Usually, the standard errors of the two shore stations are 
equal. The system standard error, , of a time measuring 
positioning system is given as 

^1 ' ^2 " ^s • 

The system standard error, o^, of a phase comparison 
positioning system is computed as a fraction of the lane width 
so that 0 ^ = = cyw = a^, where a is the standard error of 

range in fractions of a lane (i.e., a = .1 lanes) and w is 
lane width. Then Equation II-9 reduces to 
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Nfr* <js 
S(r\ p> 




(II-IO) 


d 


rrr^s “ 


where is the system standard error. 

As seen from the above formula, the d „„ is 

’ rms 

smallest at a 90° intersection angle and becomes large as 
3 approaches 0° or 180°. 

2. Hyperbolic Systems 

As in ranging systems, the repeatable accuracy of 
hyperbolic systems is a function of intersection angle and 
random errors. Because landwidth is not constant for hyper¬ 
bolic systems, the change in lane width must also be quanti¬ 
fied. As the user moves away from the base line between the 
master and a slave unit, the lane becomes wider due to the 
divergence of the hyperbolic LOP's [Ref. 8]. This divergence 
is expressed as an expansion factor, E : 

1/sin (9-/2) j 

where 0^ is the angle between the radius vectors from the 
position at p to the master and the respective slave station 
CFigure II-7). Then the standard error of one line of 
position at p is 


CTc — cr siV E 


crsy/ 



Ca) 
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Figure II-7: A hyperbolic triad 


where a is standard error in the base line in fractions of 
a line, w is lane width and E expansion factor. 

The hyperbolic LOP's bisect the angle between the 
radius vectors from p to master station and the respective 
slave station. Therefore, the angle of intersection, B, is 

Q _ 9f 9a (b) 

^ 2 

Substituting equations (a) and (b) into II-8, becomes 


a 


rms 


CSt/ 




li 

X 


2 p 

Sin % • Sin ^ 
X 2 . 


(II-ll) 
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In a triad (three-station net) one range is common to both 
lines of position. Therefore, the correlation coefficient 
is not zero. Bigelow (1963) [Ref. 2] assumes a value for 
the correlation coefficient, of 0.33 while Swanson (1963) 
[Ref. 14] gets p = 0.4. Since the determination of this 
value is based on observations comprising a statistical 
sample, the most conservative value of may be obtained 

by using ^ = 0.4. 

3. Azimuthal Systems 

In an azimuthal system, whether it is optical or 
electronic, the lines of position are radial vectors eminating 
from each of the shore stations. The repeatability of such 
systems is dependent upon the angular resolution of the 
system, and the angle of intersection of the radial vectors. 

The errors of position depend on: 

(1) the distance, r, along the radial, 

(2) the angular resolution, a, in degrees, 

(3) the angle of intersection, 3. 

The angular error may be expressed as an arc distance perpen¬ 
dicular to the respective radial at p as 

a-C - . (a) 

57*29fc ’ 

where r is the distance along the radial, a is angular resolu¬ 
tion, and 57.296 is conversion factor from degrees to radians. 


40 





























Figure II-8: Azimuthal System Repeatability 


The two shore stations are independent; therefore, the corre¬ 
lation coefficient, is zero. 

Substituting Equation (a) into 11-7^ 


'rms “* 


51.236 




( 11 - 12 ) 


Applying the sine law to the triangle shown in Figure II-8, 
it is seen that 


_ Vx _ lo 

S\n @a Sln6» Svnt'®® 3 
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I 



where Sin [180° - (6^^ + ^ ~ 

b = baseline distance. And Equation 11-12 may be written 
as 

dipr_. ^ _ \l Sm^Q» •»• S\a’*‘6a' . 

ST-. 196 

Bigelow (1963)[Ref. 2] approximates Equation 11-13 by 
letting 

r» = Pa. = — h- 

2 - S\n P /2 

Then Equation 11-13 reduces to 


i 


(11-14) 


rons 


Svo ^ • Svo ^/l 


>frx5>«'l96 


Equation 11-14 is the approximate form of Equation 11-13. 
However, Equation 11-14 is easier to compute and the error 
introduced is negligible. For a = .03°, b = 8000 meters, 

= 80°, 6^ = 30°, comparing the equations 11-13 and 11-14: 

using 11-13 


^ _ (> 03 ) (eooo) , ^ _ 

57-136 ’ 


'^S>Wx^ 80" •+ Sxo^3>0® 


- 5-ann , 


using 11-14 



(-osUeooo) i 


— 5,5m , 


vfi: % 57 - 296 Stn 70 ° Svr\ Toy^^ 
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it is seen that the difference between Equation 11-13 and 
Equation 11-14 is negligible. 

4. Sextant Angle Positions (Three Point Fixes) 

The evaluation of repeatability for sextant angle 
positions is difficult. The mathematics involved in the 
computation are quite complex. Thus, repeatability of 
sextant angle positions is more easily evaluated by a graphi¬ 
cal analysis. For the development of an analytical solution, 
see Heinzen (1977) [Ref. 10]. 

As will be seen in later sections, it is much easier 
to derive the accuracy of sextant angle positions by applying 
the method of least squares. 


E. REPEATABILITY CONTOURS 

Using the root mean square error concept, one can con¬ 
struct a family of curves to display convenient values of 
drms terms of the system geometry. 

1. Ranging Systems 

For ranging systems, the is given by 

Equation II-10 as 


^rms 


<rs 
Sm ^ 


( 11 - 10 ) 


. Note that the intersection angle, 3, is the only 

controlling geometric factor of • Figure II-9 shows an 

example of suitable geometry for a ranging system. Mathemati¬ 
cally, it can be proven that the intersection angle, 3, 
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is equal to the angle formed by radius vectors from p to 
the slaves and also the angles S^^OD and S 2 OD. The locus of 
points having a constant d^^^ and constant 3 describes a 
circle of radius r = b/2 Sin 3 with the two shore stations 
as points on the circle. 

The distance, e, along the perpendicular bisector of 
the line connecting two shore stations to the center of the 
circle is 


e. = ^ 

Q. ton ^ 


(11-15) 


■L / o 

(since, from Figure II-9, tan 3 = -^) where b is the distance 
between shore stations S-j^ and S^. 

Using 2 Og error (approximately 95% probability 
interval),Equation II-IO may be written in the following 
form 

^rms — ^ 2.' 

<rs SCo^ 


Writing Equation 11-15 as 

<2. _ I 

^ Ttao ^ 

Figure II-IO was constructed to show the relationship of 
drms/'^s ^ function of intersection angle, 3. 

Using this graph, selected contours of constant may 

be drawn as in Figure II-11. First, plot the location 
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Figure II-9: Ranging system geometry. 

of the two shore stations at a convenient scale. Draw a 
perpendicular bisector to the line joining them. Using 
Figure II-10 determine the values of e/b for the desired 
d „ contours. From the known value of b, determine distance 
e for each contour. Lay off distance e along the perpen¬ 
dicular bisector to define the center, 0, of the desired 
constant d^„ circle. The radius of the selected contour 
is the distance from the center, point 0, to the shore 
stations. 
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Values of d 


Example II-3: A phase comparison range-range 
positioning system has standard error * .01 w (lane width). 
It operates at 2 Mhz frequency. The distance, b, between two 
shore stations is 20,000 m. 


Lane width, w = 


300,000 , 

2 X 20o0 " 


75 m , 


a »aw*75x.01*.75m. 
s 


For the 2 m contour, “ 2/.75 * 2.66. 

Enter Figure II-10 with * 2.66 which intersect the 



10® 20® 30® 40®50®60®70®80®90® 

170® 160® 150®140®130®120®110®100® 

Intersection angle, S, in degrees 


Figure II-10: For ranging systems, the graph of the 

‘^rms^^s enlarged figure, 

see Appendix B.) 
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the e/b curve. It intersects at e/b = .043. For this 
specific pair where b = 20,000 m, 

e = b (.043) = 860 m 

Using the described technique, the 2m contour 

can be drawn, and the result is shown in Figure II-11. 

Thus between the 2m d contour for 3 = and the 2m d^ ^ 

rms . rms 

contour for 3 = 95°, the d error for the described system 

will be ^ 2m. 95% of the time. 

Note that when the angle of intersection, 3 = 90°, 

drms error is minimum. Therefore, as 3 increases toward 

180° or decreases toward 0°, d becomes larger. Because 

’ rms 

the tangent of the angles greater than 90° is negative, 

e values will be negative as well. Thus, the center of 

the constant circles, for angles greater than 90°, 

will be on opposite side of the baseline. As shown in 

Figure II-11, error increases as the baseline is approached. 

Contours for d values of 3, 4 and 5 meters may be constructed 

by following the procedures outlined above. 

2. Azimuthal Systems 

For azimuthal systems, d error is given by 

' ’ rms 

Equation 11-14 as 

(4 — t ^ 

'^rms - - ' -:—J 

NjrTc5f.296 S*’n^^'op/QL 
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Figure II-ll; Repeatability contours of a ranging pair. 












where a is the angular resolution, measured in degrees, 
b is the distance between two azimuth stations and 3 is 
the intersection angle of radial vectors which is defined by 
the equation 3 = 180® - (0^^ + 62 ) (Figure II- 8 ). As with 
ranging systems, the intersection angle, 3 , is the only 
geometric factor contributing to . Constant error con¬ 

tours are obtained in a similar fashion. 

Writing the Equation 11-14 with 2 a error (approximately 
951 probability interval) as 

drms _ 1 i 

n[T (5T.296)* Sm^.Svn P/a. ' 

where a = angular resolution and b = baseline distance. 

Since e/b * 1/(2 tan 3), a graph is drawn showing d^jj^g/a»b 

and e/b as a function of intersection angle, 3 (Figure 11-12). 

Figure 11-12 provides a convenient means to obtain 

the values of e distance for a selected d if a and b 

rms 

are known. 

Example II-4: The distance, b, between two azimuth 

shore stations is 2,000 m. Angular resolution of the station 

is a^ = a 2 = . 01 ®. 

For the 2 meters d^... contour 

rms 

"^rms ^ 2 _ T 

“a^ ■ ,01 X 2000 * 
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10 20 30 40 50 100 200 


Values of angle 0 (in degrees) 

Figure 11-12: For azimuthal systems, the graph of 'hsUL -J- 

tx-'o t> 

(For enlarged figure, see Appendix B.) * 
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IV 




Enter Figure 11-12 with d „^/a*b = .1 which intersects 
^ rms 

the d /a-b curve at 44®. Follow 44° line vertically to the 
rms ^ 

e/b curve, which intersects at e/b = .52. For this pair where 
b = 2,000 m, e = 2,000 x .52 = 1040 m. Using the technique 
as described for ranging systems,the 2m error contour 

may be drawn (Figure 11-13). Other contours are computed 
in same manner. 

Note that when 6 > 90®, the tangent value is negative 

and the center of constant d ^ circle will be on the 

rms 

opposite side of the baseline. For azimuthal systems, the 

minimum d^„^ error is found at 3 = 109®. 
rms 

3. Hyperbolic Systems 

The root mean square error for hyperbolic systems 
is given by Equation II-11 as 


Cms * 



2 . 


e Cos p 


Svnf 


(II-ll) 


where a is the standard error along the baseline between the 
master and respective slave station in fractions of a lane, 
w is the lane width and 3 is the intersection angle which is 
equal to 

e + 9 

e = _i_—£ (Figure II-7). 
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Figure 11-13: Repeatability contours of an azimuthal system. 
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Equation II-ll is written with. 2a error as 


2... \/ i , ^ 2 

a 2. 2. Q. 

where the correlation coefficient, was taken as .4 
[Ref. 14], Figure 11-14 was produced to show the 
values as a function of the angle subtended by the two slave 
stations, i .e . , 23. 

A parameter, p, defined as 

p _ e,-f 6j _ 0 ^ or 6,+6i _ 2 ^ 

©I ©1 ©2 ©2. 

The parameter, p, is computed with the smaller of the two 

angles, o^ or 02 , in the denominator. Thus when p = 2 

the master station is positioned on the bisector of the angle 

subtended by two slave stations, p = 3 places master station 

on one of the two trisectors, and so on (Table II-7). 

Knowing the angle subtended by the two slave stations 

at a particular point. Figure 11-14 may be used to develop 

contours of constant d First determine the d /ow ratio 

rms rms 

for a selected * Enter Figure 11-14, for several values 

of parameter p, and read the corresponding values of angle 23. 
Using the relation between 23 and 0^^ (@ 2 ) (Table II-7), plot 
these angles, 23 and 0^^ ^ conveniently scaled chart 


c^rrns 

cr 
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Values of d 



in degrees 

Figure 11-14: In hyperbolic systems, for several values of p, 

d /crw curves. 
rms 
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p 

®1 ” «2 

2 

(28)/2 

3 

C26)/3 

4 

C28)/4 

5 

C28)/5 

6 

(28)/6 

7 

(28)/7 

8 

(28)/8 

9 

(28)/9 

0 

(28)/10 


Table II-7: Relation between 28 and 6^^ or 


55 





































with a three arm protractor. Interpolating between the points, 

draw the d contour. The curve thus determined defines the 
rms 

location of a selected d contour for the specific conditions 

rms 

of triad configuration. 

Example II-5: A hyperbolic system has standard 
error, a, equal to .01 lanes along the base line. It 
operates at a frequency of 2 Mhz. Triad configuration is 
as seen in Figure 11-15 : 


lane width. 


_ V _ 300,000 

IT 2 X 2,000 


75 m 


aw=75mx.01= .75m. 


For the 4 m d ^ contoured /aw = 4/.75 = 5.32. Enter 
Figure 11-14 with ~ 5.32. For several values of p, 

read the corresponding values of angle 2$. Determine the 
values of angles 9^ or 6^ according to Table II-7. For the 
4m contour, these values are shown in Table I1-8. Using a 
three arm protractor, the points defining the 4m contour 

may be plotted. The other contours are drawn in a similar 
manner (Figure 11-15). 


56 















































cS 



Figure 11-15: Repeatability contours of a hyperbolic system 

(a = .01 lane width and f = 2 mhz) 


57 


01 osier 





























































11 


to 

II 

P. 


CNl 

II 

a 




O 



o 

LO 

• 


Ln 

• 

04 


• 

00 




o 

MH 


VO 

rg 

O 

o 




00 

II 

II 

V) 




0 

og 

/—\ 


0 


rg 

rH 

rH 

II 

o 

CD 

0} 


v—' 


> 

oa 

rH 

rg 


og 

CO 

CD 

rH 




CtS 




U 



o 

0 


o 

LO 

> 


Ln 

• 

0 


• 

LO 

tf) 


00 





rH 


o 



o 


II 

II 

mh 





rH 





rg 

rH 

o 

11 

CD 

CD 

o 



v—/ 

+J 

oa 

rH 

rg 

c 

rg 

CD 

CD 

o 

o 

V) 














o 

T3 


O 




C7> 

o 




rH 


o 




00 

II 

II 

0 




r-» 

og 





rg 

rH 


II 

CD 

CD 

00 




c 

CO. 

rH 

rg 

•H 

og 

<D 

CD 

c 




•H 








0 




T3 



o 



O 

00 



LO 

o 

rg 


LO 

rH 

CD 

o 





11 

II 

rH 

vO 



CD 

rH 





rg 

rH 

•TO 

II 

CD 

CD 

c 


v«-/ 



00. 

rH 

rg 

rg 

CD 

CD 

oa 


CM 

to 

<u 

rH 

00 

C 

CtJ 

CD 

X 

H 



o 

O 



K) 

K) 

• • 




00 

Q 



1 

vO 

II 

11 

HH 




HH 

rH 





rg 

rH 

0 

11 

CD 

CD 

rH 


V*-/ 

V—✓ 

rO 

QO 

rH 

rg 

CJ 

rg 

CD 

CD 

H 


58 




































Ill. APPLICATION OF LEAST SQUARES TO 


HYDROGRAPHIC SURVEY POSITIONS 

A. THE PRINCIPLE OF LEAST SQUARES 

Given a set of unknown parameters to be computed from 
measured physical quantities such as distance or azimuth, 
the least squares method provides a mathematical procedure 
by which the best values for the unknown parameters may be 
obtained. 

Equations must be written to define the relationship 
between the observed and the unknown parameters. If the 
number of equations that can be written is equal to the number 
of unknowns, then a unique solution may be computed. 

However, no statement can be made about the accuracy of 
the solution. In the least square method, the number of 
equations must be greater than the number of unknowns. 

As a result of this over-determined solution, the best values 
for the unknown parameters are estimated. 

This computational procedure is referred to as a least 
squares adjustment. In application, corrections are com¬ 
puted and applied to observed quantities and these quantities 
are then said to be adjusted. 

For a given set of equations, the fundamental condition 
of the least square technique requires that the sum of the 
squares of the residuals be minimized. A residual is defined 
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as the difference between an oberved value of a quantity 
and the arithmetical mean value of that quantity obtained 
from a number of observations. If the arithmetical mean value 
is stated by and observed value by x , the residual, v, 
is expressed as 

V = Xi - . (III-l) 

Suppose a set of observations were taken having residuals 

Vi, V 2 , Vj.. v^. Then in equation form, the fundamental 

condition of least squares is expressed as 

x\ 

^ . . . 4 . ■=. mm\cnum. , (III- 2 ) 

LmI 

T 

or in matrix form: V V = minimum. 

1 . W eighted Observations 

In general, some of the observed values may be more 
precise, and, therefore, entitled to have greater influence 
upon the result. Observations are assigned values called 
weights corresponding to their quality or worth. 

The assignment of weights to observed values is 
largely a matter of judgment. For example, if one set of 
measurements of a distance was made with four repetitions 
and another was made with eight repetitions, the mean of the 
second set of observations may be given twice the weight 
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of the first set. Or, when measuring angles in azimuth angle 
positions, the atmosphere may be so unsteady during one obser¬ 
vation that the observer arbitrarily assigns a weight of 
one half. 

As a general rule, if a standard error, a, has been 
computed for a set of observations, then weights are usually 
estimated according to the equation 

^ JiL , (111-3) 

<r-L 

where w^ is the weight of the ith observed quantity, 
is the standard error of that observation,and k is any 
number which has the same value for all observations. 

Equation III-3 states that weights are inversely proportional 
to the square of the standard error. Usually, the weight 
corresponding to the least accurate measurement is assigned 
a value of 1 (a unit weight). Then the value of k can be 
found and the other weights computed accordingly. 

For example, consider the standard error for two 
observations where = 3, and = 1.5. Assigning w^ = 1 ^ 



- 9 — 4 
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it is found that the second observation has a weight of 4 relative 
to the first observation. 

If measured values are' to be weighted and used in a 
least squares adjustment, then the condition is that the sum 
of the weight times their corresponding squared residuals 
must be minimized, 

f\ 

1 . + +-., (III-4) 

or in the matrix form, V := rn\oimunn . 

2. Method of Least Squares Adjustment 

In the "observation equations" method of least 
squares adjustment, the observed quantities are related to 
the desired unknown quantities through formulas or functions 
which are called observation equations. 

One observation equation is written for each measure¬ 
ment, and it is assumed that observations are independent 
of each other. In order to solve for the best value of each 
unknown parameter, at least one redundant observation equation 
must be written. That is, the number of observations must 
be greater than the unknowns. 

Observation equations may be linear or higher order 
functions. Linear observation equations can be written in 
general as follows ; 
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<^^'k +t,y + cv^ ...-hk, = 

+ ^2^ ■♦■. ^ kl — ^2. 

* I * 

I * * * I 

* ! ’ ‘ ’ 

* ? * • 

; » * I : 

Qo^ ^.-»- >^0 ^ , 


(III-5) 


where a's, b*s, c's, etc. are coefficients of unknowns x, y, 

z, etc. and the k's are constants. 

Because the observations (G,, G„ .G ) are not 

12 n 

free from random errors, each G^ must be corrected by a 
residual value, v^, in order to obtain a mathematically 
correct equation system. Thus, 


■+ b, + c,l ■+ . 

Oai^* i- -»• Cii. -*•_+|ci 

' ' * , 

\ • 

' \ : ! 

, , I 

; 1 I , 

Qo + bfijj C-rj-t+■-- —- 


= Sx -fVa 

1 

\ 

t ' 

= So-tVn . 


(III-6) 


Introducing a new notation = G^ - k^, “ ^2 ’ ^ 

etc., the following equation is obtained: 
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Oi + t.y + c,x 4 - .. 


=V, 

Olt + bay + C2.-Z. +..-<2. =:Vi 

1 ■ ; I I Uj--*--/; 

: : : : : 

’ ’ ’ t 

’ ’ I I . 

QnK. + bny + c„-£ 1-._4’„ =Vn ^ 

or in the matrix form, V = AX - L (III-8) 

This equation is called the observation equation or observa¬ 
tion equation matrix, where 


* V, ■ 


Qi ^ b>i , c-i.rr> 1 

Vi 

« A - 

Qi > ba , Ca - m2. 

' 1 1 i 


n m 

: ! ; i 

Vn 


' ' ' ! 

^ bn , Co- rnn 







L 

- 

4 

A 

X 

1 

n i 


h 

( 

\ 



L 
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In the above matrices, the subscript n denotes the number of 
observations and m denotes the number of unknowns. 

For a group of equally weighted observations, recall 
that the following condition must be enforced in order to 
perform a least square adjustment: 

n ^ 

= rT»\oimum , 

Ui 

or in the matrix form, 

T . . 

V V = minimum . 

Substituting the value for the V matrix from the observation 
Equation III-8 where V = AX - L , 

= [(^y.-o c (Axf = ] 

= ( LT) t (-from motri* oLgebra) 

= KVAA - A’'A'L - CAA +Cl_ 

and from matrix algebra, \_7A.K = 
then 

VW = - a x^a"l +Cu . 
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The minimum of this function can be found by taking 
the partial derivatives of the function with respect to each 
unknown or with respect to the X matrix (which contains all 
of the unknowns) and equating it to zero, i,e.: 

— = 0 . 

Dt 

Dividing by 2, the following result is obtained: 

- A'L =0. ) 


This is called the normal equation. 

the normal equation (III-9) becomes 

In conventional notation. 

Caa] -K, ^ Cobly ^ Cocl-i _ 

-Ca^] =0 

[V>al % ^ CWbly + Ctc3'i ^. 

.-tbj?] =0 

Cc a 3 X 4 Cc.bly ^ Cc.cl'i ^ 

' ! ' 

I ' I 

.-CoU -0 

! 

i i ; 

+Lnc.l-t4.. 

T 

.-Cnia -0 , 

where the symbol [ ] denotes the sum of the products, i.e.. 

[aa] = + a^a^ + a^a^ + . + 

+ b-a. + ba-+ . + ba . 

2 2 e 3 n n 

Vn- “ ‘>1^1 
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T 

In Equation III-9, A A is the matrix of normal 

equation coefficients of the unknowns. Multiplying Equation 
T -1 

III-9 by (A A) and reducing, the solution is obtained. 




Equation III-IO is the basic least squares matrix equation 
for equally weighted observations. The matrix X consist of 
best values for the unknowns x, y, z, etc. 

For a system of weighted observations the funda¬ 
mental condition is 


^ i = rr\»oi rr\orn , 

L=i 


or in the matrix form , \i = m mi mum . 

The normal equation matrix is derived similarly to 
the unweighted case. 


A'^na/AX -A'xa/l =0, 


(III-ll) 


or in conventional notationj 
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C vNaa 1 X 


C 1 


- CvA/a = o 
-Cwbfl =0 


-+• 

L^Wq3% -t- CIvs/V>\j3 

f ^ 

» I 

t I 

i I 

Lv^noly. ^ (l\A/nb3 ^ 

In Equation III-ll the matrices are identical to those of 
the equally weighted equation, with the addition of the 
matrix, V/, which is a diagonal nxn matrix. 

In detail, becomes 


_Cwn/Pl =0 . 


VVj 0 


0 


0 


w = 


0 


0 

0 


NNi 0 0 

0 0 

0 0 va/4 


where according to Equation III-3, 


(III-12) 



The best values of unknowns are obtained by solving 
Equation III-ll as 
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Y. = ( S' 


(III-13) 


From the combination of Equations III-8 and III-9 or III-8 
and III-ll,it is seen that 

( V i- LV- ^ O or 

K\N (V-t-L') - A^VdL - 0. 

Therefore, 

A^V = 0 or = O. CIII-14) 


Equation III-14 can be used as a check on the computation. 

Example As an elementary example illustrating 

the method of least squares adjustment by the observation 
equation method, consider the following equally weighted 
observations: 

'Xv.i s\o 

— 2.Y.a +3 y>3 ■=S 
’^'*>1 + - iY . 3=:3 

-•>^1 - = - 6 . 


^The numerical values of this example problem were taken 
from Ref. 17, page 517. 
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These four equations relate the three unknowns x^, and 
Xj to the observations. 

By including residuals, the equations may be 
rewritten as observation equations as follows; 

*1 -h 3 -t- = \0 -t Vi 

'll -Iy.2 -t-3y.3=5+Va 

or in matrix form. 


where 




- 3 3^1 ± ■> 


A- 


Q. 3 i 

1 -1 3 

T I -1 
-i -1 -i 





10 


V, 







II 


j 

5 


'Iz 






^^3 







- 



-6 
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The normal equation is 


A X - L r: O ; 


A'A zr 


2 11-1 
3-2 1. -i 

i 3 -i -i 


A"l = 


1 

1 

1 

-1 


3 i 
-2 3 

i -2 
-1 -1 





- 

- 



- 

p 





— — 



2 

i 

7 



10 


51 

3 

-2 

i 

-i 


s 

3 


2S 

1 

3 

-2 

-i 


-6 


25 

- 







- 


55 

12 

-8 

12 

15 

-4 

-8 

-4 

IS 




55 to. -e 

12 15 -4 

-8 -4 15 


Ka 

^2 


51 

29 

25 


= 0 


-1 


And the solution is X = (A^A^ A^I- J 


(AV')'* 


,022859 _.oifei87 
-.OJfelGT .08 3233 
. 007875 *0135623 


. 00 7874 8 

.0135623 

,074483 
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.022 853 
-.014481- 
. 00T815 


- .016487 
.082232 
.0125622 


, 0078>^B 
♦ 0»3 5623 
. 07‘t^S2 


Si 

29 

25 


. 94 fcl 


X = 


i-9l'o9 

2 . 46^88 


Thus the best values for the unknown parameters and 

Xj are = .9161, ^2 - 1.91109 and x^ = 2.66488. 

This computation was performed by requiring that 
T 

V V = minimum. Thus, when the best values are used in the 
equation V = AX - L, the resulting minimized residuals can 
be found. If the minimized residuals are applied to the 
observations then the observations are said to be adjusted. 



2 3 1 


-3141 


10 


.13035 


1-2 3 

T 1 -1 


i.9llo9 

— 

S 

3 


'0885 4 

-'059T 


-1 -i -i 


l.UkS 


-6 


-50T93 
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23035 


V = 


. 08 85^» 


-.0531 
. 50733 

Then adjusted observations are = 10.23035, G 2 = 5.08856, 


Gj = 2.9403 and G^ = -5.49207. 

T 

Computational check: A V must be equal to zero 
according to Equation III-14 . 


2 i ? -i 


A"V ^ 


3 

1 


«2 i -1 

3 -2 -I 


. 23 035 


.» 0 00 

. 0B656 


.000 

—OS9T 



* S013J 


.000 


According to the theory of probability, the 
above values of , X 2 and x^ have the highest probability 
of occurrence. 

Example III-2: Suppose the constant terms 10, 5, 
3, and -6 of the observation equations of Example III-l 
represent measurements having relative weights of 1, 2, 2, 
and 3, respectively. Using weighted least squares, best 
values for x^^, X 2 and Xj will be calculated. 
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The observation equations in Example III-l were 


1 ■Ai -t = \0-tVi 

tx - 2 A2 + 3x3 = 5 +>^2 
7X| -♦•'X2 -1X3 

- Xi - - X. 3 s -fc -fVit ^ 


or in the matrix form^ 


V = AX - L , 


where 




13 1 

1 -1 S 

T i -2 

-i -i -1 




X| 

Aj. 

X 3 


. L.= 


»0 

3 

3 

-6 


V = 


V, 

Vz 

^<3 


The normal equation for weighted observations is 

- A^nNL =0, 


where weight matrix N/s^ is a diagonal matrix of weights as 
follows: r 


N/y 3 : 


10 0 0 
0 1 0 o 
0 0 2 0 
0 0 0 3 
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A^Vs/L- 


l -1 i -i 
i 3 -1 -i 


a i f -i 
3 -a i -i 
i 3-1-1 


n 

*■ -1 

* 


10 0 0 



0200 



0020 



0 0 0 3 



0. 3 

i ^2 
1 


i 0 0 0 
0 2 C 0 
<5010 

0 o o 3 


" 




X 


lOT 

19 

3 


»9 

11 

-2 




-1 


-\7 

-10 

- 

10 


90 


5 


34 


3 




-6 


^ 6> 

j 


onci 

- A'^nA/L 


lOT »9 -17 

19 21 -10 

-17 -10 30 



Xi 


1 

u> 

0 

_1 


Ka 

- 

34 


X3 


^6 




The solution is y. = (A'^vVA A\VL 


\ - 


X = 


on 383 S _-ooSl 3 l^ , 0 0 37 ^ 
•.oo3\iVif ,0593795 , 01518S 

. 00 3fi( .0\5»S5i< ,0^05147 


. SlOi 
1.9856 
1/H66 


90 

3 ^ 

46 


or %»^. 9 l 0 l , =1.3856 Qnl K 5 = l.Ti 66 
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Residuals are found using Equation III-8: 


V=^X-L= 


r— •** 

2. 3 1 




ko 


. 5136 



-9101 





i -1 3 


1,9356 


5 


> c>98f 

> i -2 


2.1-\U 


3 


-.0063 

-1 -i -1 






. 3T7> 

check: 





must be 


zero. 




X i T -J. 


i 0 0 0 

3 -1 i -1 


0 1 0 0 

0 0 2 0 

i 3 -2 -i 


O 

a 

o 





* 

- 5136 


.005 

•0987 



-.00 69 


.000 

.37-?7 


- ooo 


3. Higher Order Functions 

The observation equations presented by Equation III-8 
are linear equations. If this relationship is nonlinear, 
thus defined by a higher order function, then the observation 
equations must be linearized in order to apply the least 
square adjustment method. 

Defining the general observation equation as 
G = f(x, y), where f represents a non-linear function. 

The function must be linearized by Taylor series expansion 
or by some other method. The best values of x and y can be 
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regarded as the sum of an approximate value and a 

small correction Ax, Ay. Therefore x = x^ + Ax and 
y = y^ + Ay and the above function is written in.the following 
form 


G - fCXo + Ax, yo + Ay). 

Using Taylor series expansion the observation equations may 
be linearized. 


Gi = ^(y.oj9o^ i- Ay. 

'^'^0 


Higher order •Verms 

-&yo 


( 111 - 14 ) 


The higher order terms in the series are neglected and only 
the zero and first order terms are maintained. 

After linearization, the observation equations 


become 




^ .y. ^ ^ -V- ^ G> 


Va = 


l>y^Q 


A 3 q- ^-3. 


— S' 


(III-IS) 


'n 


I 


A3 fin , 

'5'Ao ';>yo 
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or in the matrix form, V = AX - L, where 




"Dyo 



I 

I 

1 



1 

I 

\ 


\ 








L = 


-f 

, 4 

I * 


V, 

Vi 

I 


I 

! 



The remainder of the least square procedure is the same as 
indicated by Equations III-9, III-IO or III-ll, III-13. 

In the linearization process, the higher order terms 
were neglected. For this assumption to be valid. Ax and 
Ay should be small so that their products in the series 
expansion approach zero.(Ax • Ay = 0). This can be achieved 
only if the values of and y^ are very close to the values 
of X and y. Therefore, x^ and y^ must be precomputed, or 
the original assumed x^ and y^ must be improved by successive 
iterations until the adjusted observations equal the measured 
values. 
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4. Equations for the Precision of Adjusted Quantities 

After calculating the best values of the unknowns, 
or X matrix, the V matrix,or the adjustments to the observations, 
can be computed from the observation equation which is 
V = AX - L, whether the observations are weighted or not. 


Using the V matrix, the standard error of an obser¬ 


vation of unit weight is given by the following equations 
[Ref. 17] : 


for unweighted observations, 



CIII-16a) 


for weighted observations 



(III-16b) 


where 


Oq is the standard error of an observation which has unit 
weight, 


n is the number of observations, 
m is the number of unknowns. 
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Standard errors of the best values for the unknowns are 
then given by the following equation: 





CIII-17) 


where 

is the standard error of the ith adjusted quantity, 

e.g., the quantity in the ith row of the X matrix, 

a is the standard error of unit weight as found by 
o 

Equation III-16a or III-16b, 

T -1 

q^^ is an element of, for unweighted case, (A A) 

T -1 

or, for weighted case, (A WA) matrix. 

T -1 T -1 

If the (A A) or (A WA) matrices are written in detailed 
form as 

(/("Af or = 


then the standard errors of the best values of the individual 
adjusted quantl^ties are: 

Oi (J'o'J , 


- 

Sal R23- 

S31 S32. 

! I ' 
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— CTo \j C|^2.' , 

^3 = CTo 4 ^3? ' , 

Example III-3: The standard errors of the best 
values for , x^ and x^ in Example III-l . 

Standard error of unit weight for unweighted 
observation is 


CTo = 


n-m 




13035 .08956 -.00337 .50793 


.313 




319 

^-3 


56S , 


23035 ' 
08B56 
.00597 
507-93 
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The standard errors of the best values are given by 


Equation III-17 as <>■<; . 

For unweighted observations, q^^j^s are the elements 
T -1 

of CA A) which was calculated in Example III-l. 




.0 21859 
--OlfclSl 
. OOJ81-5 


.0OT8>5 

. 06-^133 . 0135(^13 

. 0\35bl3 . 0?4M-83 


= - 565 n/ .011859~‘ ^,085, 

<y%2, = CTo^l . S65 0S32.33 

cr%3 = o-oNT^ = . S4S \/.0T4483’ = .1^^. 


T -1 

In the (A A) matrix, off diagonal terms are used 
to find the covariances of unknowns. Covariance 0^2 
is equal to ^ 

Or^^ = CTc \/c|,3^ 

From covariances, the correlation coefficients of variables 
are obtained. Correlation coefficient 5^2 given as 


e 


12 


O',! 


0;o-! • 
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mA a 


The interpretation of the standard errors computed 
above is that there is a 68% probability that the adjusted 
values for and x^ are within ±.085, ±.163 and ±.154 

of their true values, respectively. 

Example III-4: The standard errors of x^, x^ and 
Xj in Example III-2. 

The standard error of unit weight for weighted 
observations is 


o-o =\/:£!^ 
n-m 


- 


S\ZG .098T-. 0069 


1 0 0 o 



0 1 0 0 


^ogfi> 

0 0 2 0 


-.0063 

0 0 0 3 


^mi- 

• 


^ -m 


\ 






n-m 



. 
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The standard errors of best values are given as 


O-L = O-o \/S CL' . 

T -1 

For weighted observations, are the elements of (A WA) 

which has been already calculated in Example III-2. 




o 011^839 
— 00 81314 

. OO 3*^4 


-.00 8\3\4- 

. 05 9 37-35 
, 0\5l854 


. 00374 
, 0»5I8S4 
. 040514 


(J^ = CTo i \/v0l\3839 ' _ .0S99 , 

cr.. =.8H^\/-o59i>9?=.2os4, 

- cr, ^ . 843 \|.oifos(4' = - \m. 


84 













































B. APPLICATION OF LEAST SQUARES TO HYDROGRAPHIC 

POSITIONING SYSTEMS 

If redundant data are available, the least square adjust¬ 
ment method may be used to compute the coordinates of hydro- 
graphic survey positions. Observation equations may be 
written for various types of survey methods. By expressing 
these equations in matrix notation and using successive approxi¬ 
mations of the unknowns, the best values for the coordinates 
of survey positions may be determined. The predictable 
accuracy of these best values may also be foimd. Thus, 
redundant observations, coupled with mathematical data 
adjustment techniques, produce a viable method of system 
calibration for hydrographic survey data. This method of 
calibration is referred to as auto calubration. 

1 . Azimuth Angle Positions 

The working range of azimuthal systems is limited to 
line of sight distances, i.e., 5-15^nautical miles, depending 
upon the height of the observing instrument. 

Because of this range limit, the Universal Transverse 
Mercator (UTM), or other plane coordinate systems, may be used. 

Let 

^1’ ^2’ ^3 * Northings of the shore stations 1, 2 and 3, 
respectively, 

Xj^, x^, Xj = Eastings of the shore stations 1, 2 and 3, 
respectively, 

P = The position of the survey vessel. 
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Then, the azimuth (from north) of the survey vessel from 
shore stations can be written in terms of coordinates as 


~ j Alps: tan"' y A3P = tan"*' 

~ Xi. y. p-yo. y-p - 


In these equations, Xp and Xp are the best estimate of the 
survey vessel coordinates which are to be determined. 

These equations are non-linear. Thus, in order to 
form observation equations, they must be linearized. 
Letting ~ ^ Ay, where and y^ are 

the approximate coordinates of the vessel’s position, and 
using Taylor series expansion for linearization. 


f(K^y) ()^o>S)o> -I- ^ Aj-h.^ 

where .f = A Lo ^ -tan"' . 

The partial derivatives are 

90 

The observation equations now may be written in the follow¬ 
ing detailed form: 
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yo-y> 

%o-^» 





S?b 

si 




Aip" Avo 

V| 




/ill 


Vi 

% 

— c\ 

ao-Sa. 





Alp - A:/o 

- p 






/\3p>_ A 30 



ao-a3 













1 


where 

A to 


•ban"' 




and Alp 5 Aap j Asp 
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are the measured azimuths, and g = 57.2958, the conversion 
factor from radians to degrees. 

Having obtained the observation equation, the normal 
equation may be formed and solved by following the procedures 
outlined in Section II.A. 

Using the computed values of Ax and Ay, new trial 
point coordinates may be formed as follows: 

i) = -t-Ay . 

The values are substituted in the observation equation for 
the initial x^, y^ coordinates. The least square solution 
is iterated until the Ax and Ay values approach zero. 

Example III-5: Referring to Figure III-2, the 
coordinates of the shore stations are 


Luces (#1) 

X 4,055,042.7 m 
y 595,794.5 m 


Mussel (#2) 
4,053,453.2 m 
597,967.8 m 


MB4 (#3) 
4,053,917.2 m 
603,425.2 m 


The standard errors for the azimuth observations are = .02°, 

O 2 = .024° and = .018°. The following angles were 
measured: 

p - Luces - Mussel = = 50?164 
p - Mussel - Luces ~ ^2 ~ ^^*360 
p - MB4 - Mussel = = 47?865 
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Figure III-2: Determination of a position for azimuthal 

systems using the least square method. 
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The least square method will be used to determine the best 
values for the coordinates of the survey vessel. 

Given; = 126?180 Given: = 306?180 

Measured: ^ 50?164 Measured: a 2 = 99?360 

Alp « 76?016 A^p = 45?540 

Given: A 22 “ 265?140 
Measured: = 47?865 

A.^ = 313?005 

3P 

First, assume = 4,055,000 

y© = 600,000 


The observation equation, V = AX - L, is then 


p 







■ 

V, 


m 

0 

• 

1 





-14.56fe 

Va 

— 

-. o\i-8S 

.0l358> 



— 


% 


• 0\S208 

. 00 ‘fSDl- 




25.441 






“ <«■ 




90 







































Using the standard errors of each station, a weight matrix 
is formed: 



0 



Let k = 0.024, then 


0 0 


V7 = 


0 



i 0 

0 Ub 


Using the A, L and W matrices, the components of the 

T T 

normal equation, A WAX - A WL = 0, are written 







— “ 

. 000997H 

-.0001091- 




i.\o31 

.-000109T- 

.00021S^ 




.1131 
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The solution, .X = (A^WA) ^ (A^WL) , is then 


1059.2 

514.6 





SI4.6 

i)679.M 


.^32 


1194.0 


Now, the new trial point coordinates are found. 

% = 055,000 d-\a3\-8 ^,056^231.8, 

3 = yo-TAvj = 600,000 


Using new trial point coordinates, solutions are repeated 
until Ax and Ay vanish. Table III-l shows the data for other 
trial Ax and Ay values and the new trial point coordinates. 

The best estimate of the coordinates for the survey 
vessel is 

Xp = 4,056,302.9 

y = 600,867.4 

P 

2. Sextant Angle Positions 

In sextant angle positions, similar to the resection 
problem in geodetic work, the measured quantities are the 
included angles at the sounding vessel between the shore 
stations as shown in Figure III-3. 
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Iteration Change in Coordinates Coordinates 
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Plane coordinates are again used because of the 
visual range limitation. 

It can be seen from Figure III-3 that 

V ■ ^Pl “ “l ' 

^p3 ■ V “ “2 ’ 

V ■ ^3 “ “3 » 
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or in terms o£ the coordinates 


too-* _ too-' = =<1 , 

tan'' =<=<2., 

ton-' ^P _tan-' =o<3. 

Xit —Xp 

Letting x_ = x + Ax and - Y + Ay and linearizing with 

po p o 

Taylor series expansion, 

f('k,a') = ^ Ax. + +- 

where 

_ 9x-bJo _ y'-bio 

^ \o 

r=. _ 'A a — 

"Dyo s,^ 
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The observation equations may then be expressed as 










ya-iJo 

y»-yo 

7-v->.o 




V. 







Ax 


c>^[~ i Ao\^ 
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9'i-ye 

, ) 


>-3 - 1^0 









^to 













Ay 


<^’^3—(Apt, "Aoii 

V3 




H-'ko 







C ^ 








where 

$ = 57.2958 is the conversion factor from radians to 
degrees, 

^01’ ^02* ^03* '^04 computed azimuths of lines 

01, 02, 03 , 04 using trial point coordinates x^, y^. 

Once establishing the observation equations, the 
solution is found as 




The process is repeated until Ax and Ay become very 

small. 

Ejiample III-6: Referring to Figure III-4, the 
coordinates of the shore station are 
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Figure III-4; Determination of a position for sextant angle 

fixes using least squares adjustment. 
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MB4 (#1) 


Use C^2) 


Mussel (#5) 


Luces (#4) 


X 4,053,917.2 4,051,216.9 4,053,453.2 4,055,042.7 

y 603,425.2 600,372.0 597,967.8 595,794.5 

Measured angles are 
MB4 - p - Use = 49?927^ 

Use-p - Mussel = 38?130, 

Mussel - p - Luces = 30.396. 

The least square method will be used to determine 
the best values for the coordinates of the survey vessel. 

Let the first assumed position be = 4,057,000 and = 599,000. 
Using the first approximate position. 



9 


= 124!862, = 165?712, A^^ = 196?235 and A^^ = 238?591 

are obtained. 

The observation equation is 


V, 


—. 00 6366 

- 00 3834 




3”. Of? 

Va 


OO668T 

. 00 4984 



— 

T. fc0> 

V3 


— 00 86S3 

-.00 694-1 


Ay 

— w 


-a% ifco 


and normal equation, A AX - A L = 0, is 
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j oo o lUofe 

.OOOOO 15 




-.00301- 

. 0000015 

.0000 


Ay 
^ — 


* 15114 


The solution, X = (A^A) ^ A^L, is 


6119,4 

- 1116 


-.oo3o> 


-4T-0 

-IIT , 6) 

1140?. 6> 


• I5T-H 




Then, the new trial point coordinates are 

X = + AX = 4,057,000 + (-47) = 4,056,953, 

y = + Ay = 599,000 + 1793.1 = 600,793.1. 

Using new trial point coordinates, the above steps are 
repeated until Ax and Ay values become vanishingly small. 

For every trial Ax and Ay value, new trial points coordinates 
are tabulated in Table III-2. 

The best values for the coordinates of the sounding 
vessel are 

Xp = 4,056,512.3, 

y^ = 600,864.5 , 
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Iteration Change in Coordinates Coordinates 

No. , , 
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Table III-2: For least squares solution, successive iterations 
applied to sextant angle positions 
























3. Range-Range Positions 


In range-range positioning, when the distances are 
short (i.e., line o£ sight type equipment, less than 20 
nautical miles), a plane coordinate system may be used. Let 
x^, x^, = Eastings of the shore stations #1, 2, 3 

^1’ ^2’ ^3 ~ Northings of the shore stations #1, 2, 3 

Xp = Easting of the survey vessel 

y^ = Northing of the survey vessel 

Then, the distance between the ith shore station and 
the survey vessel, in plane coordinates, is 


Sep = ^p- ^ \ 


This function is non-linear and has to be linearized. 
Introducing approximate coordinates (x^, y^) for x^ and 



then 



= AX , 

+ Ay 


Using Taylor series for linearization, the result becomes 


-f -^11 Ay + 2L Aa + 
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After linearization, the observation equation is 




'ko— 'k\ 





V. 



S VO 


An. 


1 

i 

o 

n/q 

\/3 

— 

^2.0 

^2jO 


Ay 

— 

- Sio 

^3p - S 30 












^30 





or in the matrix form, V = AX - L, 
where ; 


S^p are the measured distances, 

$ 10 , ^20’ ^30 computed distances using and y^. 

In ranging systems, when the distances are long, 
coordinate computations must be carried out on the appro¬ 
priate ellipsoid using rigorous geodetic formulas. Let 


0c ) 

lO 



= Computed geographical coordinates, latitude 
and longitude, of the survey vessel, 

= Latitude and longitude of the ith shore station, 
= Azimuth from ith shore station to approximated 
position Oj 

= Azimuth from O to ith shore station, 

= Distance between O and ith shore station. 
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Although the computed observations must utilize rigorous 
geodetic solutions, the differential equations of the observa¬ 
tions may be approximated using spherical trigonometry [Ref. 3]: 

dSoL = U Cos Aol - Rc Cos A Co cJ(t)L 

+ NcCosC|)l Sin Alo ( AXo - AXu^ 1, 

where and are in seconds of arc, dS in meters. Sin 1’* 
is the‘conversion factor from seconds to radians. and R. 

are the radius of curvature in the plane of meridian at 
point O and ith shore station, respectively, defined as 
[Ref. 18] 

R. = o (I- e.^) 

where a is the semi major axis of the datum ellipsoid, 

2 

and e is the eccentricity of the datum ellipsoid. 

and are the radius of curvature in the plane of prime 
vertical at point O and at ith shore station, respectively, 
defined as [Ref. 18] 

N — ^ _ — 

is the latitude of point O and 0^1^ latitude of the ith shore 

station. 
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The partial derivatives of computed observations 
with respect to parameters are 

^ _ Sin i" Ro Cos Aol > 

D0O 

^ Sml'Mi. Cos0i:S>.aA.Co. 

"ZiXo 

Then, observation equation is 


V* 


_ i^oCos Ao» , Cos0i Slr\-Aio 

Va 

= Sinl 

_ Ro Cos Aoa * Hx Sin-Axo 

V 3 


— Ro Cos Ao 3 . ^2 GjsCt)^ Sin A 30 

L 


A(|) 

AX 


Sp» - Sov 
Spa - Soa 
3p3 - Soi 


where , S^2» ^03 computed distances using inverse 
distance and azimuth formulas (these formulas could be found 
in any geodesy text)*, and measured distances 

between point p and the respective shore station. 

After forming the observation equation, the normal 
equation is found and solved as in previous examples. This 
process is repeated until Acj) and AX become smaller than the 
resolution of the positioning system. 
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Example III-7: Referring to Figure III-5, the coordi¬ 
nates of the shore stations are 

Luces (^1) Mussel (#2) MB4 (#3) 

X 4,055,042.7 4,053,453.2 4,053,917.2 

y 595,794.5 597,967.8 603,425.2 

Using the least squares procedure, best values of 
coordinates of the vessel may be found. Let the first 
assumed position x^ = 4,056,000 m and y^ = 598,000 m. 

Measured distances are p - LUCES = 4350 m, p - MUSSEL = 4506 m, 
and p - MB4 = 5267 m. For the first approximate position of 
the vessel, the observation equation is written: 


V| 


.939 

. oat 


Av. 


1303.0 

Vx 
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.353 

.9 a 
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.3)53 
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T T 

Normal equation, A AX - A L = 0, is 


i.28T- 

.042 


Ay. 


^ m 

ISiH .8 
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Figure III-5; Determination of a position for range-range 

systems using least squares adjustment. 
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And solution, X = (A^A)'^ A^L, is 


A^’ 



-.019 


2834.8 


2l83.,l 

Ay 


-.019 

.584 




TS1-.4 


Then, new trial point coordinates are: 

X = + Ax = 4,056,000 + 2183.2 = 4,058,183.2 

y = Yq + Ay = 598,000 + 787.4 = 598,787.4 

Using the new trial point coordinates, the above steps are 
repeated until Ax and Ay values become smaller than the 
system resolution. For every trial, the change in coordi¬ 
nates and the coordinates of new trial points are tabulated 
in Table III-3. 

The best values for the coordinates of the sounding 
vessel are 

x„ = 4,057,501.2 

P 

yp = 599,567.7 


The standard errors in the northing and easting may 
also be calculated; 

\} ■— ( ^or A QnA V-. ■) Lovt tVe.rc\tvc»'» '/aLues ore used ^, 




. S2>o 

»5Hb 
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-3.1 
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- 1.6 
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Iteration Change in Coordinates Coordinates 
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= 1 ^ J 
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’ 2 . 4 ' 

- 

1 . 1 - -i.S 






-1-5 


cr© 




4.0^ ; 



T -1 

where are the elements o£ (A A) which has been 

calculated as 



-.14 

.T4 


<J>f. cTo r= 4,04 ^ -64^ = 13 m , 

CTjj =: CTc \|^j^ = 4.04 'I -46 =3). 5201^ 


4. Hyperbolic Positioning Systems 

Hyperbolic positioning systems measure the difference 
in distance from a vessel to the two shore stations. In 
Figure III-6, station number 2 is the master station, and 
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Figure iII-6: Determination of a position for hyperbolic 

systems using least squares adjustment 
method. 
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numbers 1, 3 and 4 are slaves. Point p is the vessel’s 
position, and its coordinates are designated as (f>p and Xp. 

Point O is the first approximate position, with ip^ and X^ 
representing its coordinates. 

The differential equations of the computed distances 
to each station, may be written as [Ref. 3]: 

dsoC = Sfni" C-Bo Co=AoL i0o- foCos Aloilpc 

Nc Cos (f) L Sfn. A Co ^ ^ ^ ^ C"3 

where i represents the shore station number. 

represents the azimuth from the ith shore station 
to approximate position 0. 

is the radius of curvature in the plane of 
meridian at point O (as defined in Section B.3.). 

is the radius of curvature in the plane prime 
vertical at the ith shore station (as defined in Section B.3.). 
The partial derivatives of the function with respect 

to d)^ and X_^ are 

o o 


^i£L ^ « sco. i" Cos AoC, 

-500 


Srn,i''Cos0)c$(a.ALo. 

Ao 
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The range difference between the distance from the 


vessel to the master and the distance from the vessel to 
the respective slave station is expressed in the equations 
below. 


l!si _ = Sfru i" ( Cos ^oi - Cos 

^2)00 0o 

__ _bSo(. SCri/Xio^ 

Xo ^Xo 

Note that station number 2 is the master station, and 
the range difference is stated in terms of the partial 
derivatives. 

With this information, the observation equation is 
written as: 


Vt 


po ( C-os Ao» — C05. Aox') ^ Cos(t^i 1 Sin, Av2o- S\Vi Xio^ 


= Sioi“ 

^0 (G>s Ao 3 -G>s Xoi , His. Cos eta ( Sio. A20 - SCfv l\3^ 

V3 


Ro CCcS A04-Cxs Ao 2 , H3 GasCj)., ( S\fv A20 -SsO-Xm) 

L -ri 


A(J) 

AX 


( S^x-Sp\) -iSox- Soi') 

( Spz — Sp3^— ( So*!. - S03) 

( S^i — Spit') —(Soa— Son!) 
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where : 

A^l, computed azimuths from approximate 

position 0 to shore station 1, 2 ,... 

Aijj, A^ij,... are computed azimuths from shore station 
number 1, 2,... to approximate position 0 . 

(Sp 2 " CSp 2 - measured range 

differences, 

^^02 ' ^01^’ ^^02 ' ^03^’ •** computed range 

differences . 

Sin !•* is conversion factor from second to radian. 

After writing the observation equation, the normal 
equation is solved and the best estimate of the coordinate 
values is found as previously discussed. 

The process is iterated until AcJ) and AX become 
smaller than the standard error of the specific hyperbolic 
system be'ing used. 

5. Global Positioning System (GPS) 

Global Positioning System fixes are obtained 
utilizing the computed distances from the position of GSP 
satellites to a GPS receiver. The receiver measures the 
arrival of a timing pulse from every satellite within 
acquisition range. The transmit time of each pulse is 
encoded in the received signal. Thus, distance is computed 
using the one way travel time between each satellite and the 
receiver multiplied by the propagation velocity of electro¬ 
magnetic energy. Three such satellite to receiver ranges 


113 





















may then be applied to solve for the coordinates of the 


receiver. 

Using three satellites to determine a fix results in 
a unique solution for the position coordinates (x, y, z). 
However, significant error may be induced due to drift in 
the receiver clock. This additional unknown, receiver clock 
bias CE), may be resolved by processing four satellite 
ranges. 


For position fixing at sea,it is likely that the 


z coordinate may be input as a known value based on a given 
antenna height above sea level. Thus, the number of unknowns 
will be reduced to three. By using four or more satellites, 
redundant observations are then available so that the data 
can be adjusted by the method of least squares. 

Introducing the following variables, observation 
equations may be written in a straightforward manner: 

, R 2 , Rj ,..• = Measured distances from receiver 

to satellites S^, S^, S^, ... 

(Xi, y^, z^), (X 2 , y 2 > ^ 2 ^* “ Known positions of 

satellites S^, S^,... 

X, y, z = Unknown position of the observer,p. 

E = Receiver clock bias (unknown). 

Then, the basic equations are 
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^ E ^ 



Here, the ranges , R 2 , R^j ♦•• * include the actual 
satellite to receiver distance plus some offset due to 
receiver clock error. In the above equations, the 
satellite positions are known, and the four unknowns are the 
user position (x, y, z) and user clock error. 


Since the observation equations are non-linear, the 


Taylor series must be applied to form equations suitable for 
use with the method of least squares. Let 


X = Xq + Ax 


z = Zq Az 


E = + AE. 

o 


y = Yq + Ay 


Using Taylor series, 


Rip ^ Eo + 




where R. is the distance between a satellite and the user 
iP 

position, p. 
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^io distance between a satellite and approxi¬ 

mated user position. 

Observation equations are written in the following 
detailed matrix form: 
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where 

Rfp, R 2 P . are the measured distances, 

RfO, R 2 Q . are computed ranges from the formula 



From this matrix, the normal equation may be formed 
and solved as previously discussed. The process is repeated 
until the values of Ax, Ay, Az and AE approach zero. 
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Example III- 8 : At 0800 Zulu, May 1 , 1980, a satellite 
fix was taken using a GPS receiver aboard USNS ACANIA in 
Monterey Bay. The measured distances between the satellites 
and the receiver were 

Spi - 20,640,380.8 m 
Sp 2 = 20,357,184.1 m 
Sp 3 = 23,287,346.8 m 
= 21,699,908.4 m 
Sp 5 = 25,416,133.6 m 


The satellite 
S #1 

x^ = 6,097,294.4 

y^ = -4,364,543.9 
= 22,658,876.2 


coordinates were 
S »2 

= 1,819,274.3 
y 2 = -2,240,846.4 
Z 2 = 23,721,192.7 


S #3 

x^ = 9,268,094.7 
y^ = 13,290,138.0 
Z 3 = 13,622,934.4 


S #4 S #5 

x^ = -8,198,461.9 Xg = -21,419,309.7 

y^ = -18,813,603.1 y^ = 12,865,351.8 

Z 4 = 19,040,626.8 z^ = 4,832,143.1 


Applying the method of least squares to determine the 
user position, first assume: 
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= -2,640,000 
= -4,235,000 
2 ^ = 3,960,000 

= 10,000 . 

o 

For the first iteration, the observation equation 
is written as 


V, 
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and the normal equation, AAX-AL=0, 


i.0S9 
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The solution, X = (A A)’ A L, is 


30 40- 9 




'21-05.2 

-\0S^0.9 


New trial point coordinates are 


= -2,636,959.1 


= 3,957,294.8 

o 


= -4,236,646.6 


E 


o 


-840.9 


Using new trial point coordinates, the above steps 
are repeated until Ax, Ay, Az and AE values become vanish¬ 
ingly small. For other trials (iterations), the Ax, Ay, Az 
and AE values and new trial points coordinates are tabulated 
in Table III-4. 


The final user coordinates are 
X = -2,636,937.1 
y = -4,236,666.2 
z = 3,957,250.8 . 

And receiver clock bias is E = -869.8 m. 
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Change in Coordinates Coordinates 
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C. USE OF THE ERROR ELLIPSE IN ANALYZING THE ACCURACY 

OF HYDROGRAPHIC POSITIONS 

In the least square adjustment process, the positional 
errors are found in the direction of the x and y (<J) and X) 
coordinate axes. These o^ and values indicate the 

expected displacement of the fix in the direction of the 
coordinate axes, but they do not necessarily define the maxi¬ 
mum and minimum errors associated with the axes of error 
ellipse (Figure III-7]. 

Maximum and minimum standard errors are found by defining 
the orientation of the error ellipse in terms of the x,y 
coordinate system. Let the coordinate system defining the 
semi-major and semi-minor axes of the error ellipse be u 
and V as indicated in Figure III-8. 


The following relationship exists between the ellipse 
(u and v) and the ground (x and y) coordinate system; 


U •=: y. Sin 0 -hi) Cos 0 

V = y. Cose -y Sine 


(III-18) 


In these transformation equations, the angle 6 is the 
rotational angle between the y and u axes (measured clockwise 
from y axis to the u axis) . 

The lengths of the semi-major and semi-minor axes are 
given by : 
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Figure III-7: Error ellipses formed at the determined 

positions CD* B. Thomson and D. E. Wells, 
1977). 
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Figure III-8: Error Ellipse. 



0-v « 



(III-19) 


In above equations, o^, the standard error o£ unit weight, 
is known from the least square adjustment of point p, q^^ and 
qyv given by^: 

^For detailed derivation of these equations see Ref. 16, 
pages 181-183. 
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^ Cos 2.© + C|^y SvO 20 


(III-20) 


Svv —-^(Siikx'♦■ Sy|j ^ +■ "2 ^ ^ (III-21) 

T -1 

where q > Q ^ ^ v elements of the (A A) 

XX yy xy 

T -1 

(for unweighted observations) or (A WA) (for weighted 
observations), i.e.. 


(/^kT' or Lk^'HK')' 


9x> 



Syy 


Equation III-20 reaches its extreme value, and q^^ is 
maximum, when 

ton le (III-22) 

9 %>.“ 9 ^.^ 

Inserting Equation III-22 into III-20 and III-21, and 
defining D as. 

D = L ( 9 ^^ _q^y'y^+ 4(9»yf'l'f cm-23) 
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and may be written as 


± + qyy -i-D ) (III-24) 

^vv (Ry.> +9yy (III-25) 


and from Equation III-19, the semi-major and semi-minor axes are 


Q. ^ Oo (9»+qyy ^ (III-26) 

b? = i a-,;H+ q-Dy (111-27) 


Using these expressions, the error ellipse can be con¬ 
structed at any point whose coordinates were determined by 

T -1 T -1 

least square adjustment if the (A A) or (A WA) matrixes 
are known. 

Example III-9: In order to determine the error ellipse 

parameters for the range-range example problem (example III-7), 
T -1 

recall that (A A) was determined as 



-.m .!-(> 
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4.04. 


and the standard error of unit weight was = 

The semi-major and semi-minor axes of the error ellipse, 
according to Equations III-26 and III-27, are found by first 
solving Equation III-23 : 

b C (.6^1 (-.1^') J — , 305. 

Then, 

^ ~ ^ ^0 (-t"9yy ^ ^ 

(* + 3o5) = \3.9 

Q = 3. m 

= J_ -l-,T6. - .^OS) ^ 9»SH 

2 

b = 2.99 n.. 

The semi-major axis is 3.73 m, and semi-minor axis is 2.99 m. 
According to Equation III-22, the angle 9 is found; 
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ton le = _ ^ = — 

20 = 113?2 

0 = 56?6 which defines the orientation of the error 
ellipse. 

1. Some Characteristics of the Error Ellipse 

A number of important properties of the error ellipse 

can be obtained by analyzing the equations given in the previous 

section. The existence of the error ellipse points out an 

important fact that the accuracy of the location of a point 

in question is not the same in every direction. An analysis 

of equations III-26 and III-27 demonstrates that the formulas 

for the semi-major and semi-minor axes are composed of two 

parts: the standard error of unit weight, which defines the 

T -1 

scale of the error ellipse, and the elements of the (A A) or 
T -1 

(A WA) -matrix, which define its shape. 

To reduce the error ellipse into a circle, where the 
accuracy of position is equal in every direction, the follow¬ 
ing condition must be met: 

a/b = 1 . 

According to Equations III-26 and III-27 this is possible 
only if D = 0 : 
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which means that 


^xx ^yy *^xy ^ ’ 
and, according to Equation III-17, 

a = o 
X y • 

Another important characteristic is that the sum of 
the squares of the standard errors in x and y directions is 
invariant to the rotation of the coordinate system, or 

— CTa -t- Cv"" — CT^ 4- crj’ . (I II-28) 


Equation III-28 leads to the concept of root mean 
square error, , as follows ; 

= \/a7T^ (III-29) 


or, from Equations III-26 and III-27. 


rms 


^ or. 




128 




























































IV. CONCLUSION 


Conventional survey systems will provide the primary means 
of hydrographic positioning for several years to come. 

Thus, the concepts of and the graphical approach to 

developing error contours are very useful tools in survey 
planning and execution. 

Survey planners must exercise care in establishing the 
position of navigation aids. The resultant net geometry 
determines the accuracy, and thus the error, of the fix 

positions. Accuracy requirements for the collection of 
hydrographic survey data greatly limit the size of the 
effective survey area. Through careful planning, the 
number of navigation aid shore stations can be minimized 
while still meeting position accuracy requirements for the 
survey. 

Currently, more research is needed to determine the 
environmental factors which govern variations in the propa¬ 
gation velocity of electromagnetic energy. If this important 
parameter could be more accurately modeled throughout the 
survey area, the effects of systematic errors due to these 
velocity variations could be greatly minimized. 

As shown in this paper, methods exist today by which the 
accuracy of survey positions can be greatly improved through 
the use of redundant observations and data adjustment techniques. 
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The method of least squares adjustment provides a best esti¬ 
mate of position, plus the size and orientation of the error 
ellipse associated with that position, for every point deter¬ 
mined by the survey system. In addition, the error ellipse 
quantifies the predictable accuracy (as shown in Figure II-7) 
of each position as compared to the repeatable accuracy 
available from conventional survey methods. 

The application of these techniques will become more 
widespread when the Global Positioning System is fully 
operational. Observations of position from any number of 
navigation and positioning systems (GPS, LORAN, hydro position¬ 
ing systems, etc.) can be combined in a least square solution. 
Observation equations may be written and weights can be 
assigned as a function of accuracy for each system. 

In preparation for these future improvements, hydrographers 
must work to understand and implement the concepts discussed 
in this paper. Data must be processed by computer. There¬ 
fore, programs need to be written which can perform the 
iterative least squares adjustment on the appropriate obser¬ 
vation equations. Errors must be analyzed to assess the 
improvement in accuracy resulting from redundant observations. 
Additionally, standard hydrographic survey procedures need 
to be reviewed to determine if more efficient methods may 
be adopted when redundant observations and data adjustment 
techniques are used. For example, the method of least 
squares may be programmed into onboard computers so that 
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data may be adjusted and evaluated in real time. Alternately, 
redundant data may be collected and recorded for later pro¬ 
cessing ashore. 

The technology is available today to employ data adjust¬ 
ment methods in hydrography. This technology must be 
analyzed and adapted to match the systems and requirements 
unique to hydrographic survey. 
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ANALYSIS OF FANDOM ERRORS 


APPENDIX A 

(The information given in this section was taken directly 
from References 1 and 20.) 

ONE DIMENSIONAL ERRORS 

An error in a measurement is the difference between the 
"true" value of a quantity and the measured or derived value. 
The "true" value can never really be determined because of 
instrument limitations and human fallibility. In determining 
the value of a quantity, only one measurement may be necessary 
when an approximate value is sufficient. If, on the other 
hand, the quantity is important enough to require a more 
precise value, repeated measurements are made. Variations 
will exist between the values obtained from several measure¬ 
ments. Applying the theory of the normal distribution to 
these measurements, the "best" value for the quantity is the 
mean or average of all the observed values. The differences 
between the mean and the observed values are the apparent 
errors or residuals which are used to derive a statement of 
precision for the measuring process. When the residuals are 
randomly distributed about the mean, the precision of the 
measurement is expressed by a single term, the standard 
error, which is commonly designated by the Greek letter "sigma" 
(a). For a one dimensional normal distribution, this value 
is computed by squaring all the residual errors (v) , adding 
the squared values, dividing by the number of errors less 
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one (if n independent direct measurements are taken of the 
same quantity, then the first measurement establishes a 
value for the unknown and all additional measurements, 

(n-1) in number, are redundant), and taking the square root: 



where is the residual defined by the equation ^i “ ^i " 

: observed value 
X : mean value 

n : the number of observations . 

The normal distribution itself is represented by the 
function: 

pL\j) — e 

The normal distribution curve and the meaning of the standard 
errors are illustrated in Figure A-1. The central vertical 
axis, pCv), represents the probability of zero error with 
positive errors plotted to the right and negative errors 
to the left. The height of the curve above a particular 
point on the horizontal axis is proportional to the probability 
of an error of that amount. 
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Figure Al: One dimensional Normal Distribution Curve 


It can be observed from the normal distribution curve 
that the total area under the curve is equal to unity. Also, 
the area under the curve between any two values of Vj^ 
and is equal to the probability of an error occurring 
between these limits. So, to find the probability of an 
error between and P(v) has to be integrated between 
and 72. The area under the curve between the limits of 
= -a and ~ 68.27% of the total area under the 

curve. This means that there is 68.27% probability that 
errors in any further measurements made under the same 
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conditions will not exceed the standard error, o, with a 
68.27% probability. The standard error does not indicate 
the probability that an error of a certain size will occur; it 
only indicates that 68.27% of the errors will fall within 
the specified limits of plus or minus one sigma. 

If other probability levels are desired, the appropriate 
conversion factor may be found in Table Al. For example, 
for 95% probability, o should be multiplied by a linear 
error conversion factor of 2. 


Probability, % 


Linear error 
conversion factor 


50 

68.27 

90 

95 

99.7 


.6745 

1.000 

1.6449 

2.000 

3.000 


Table Al: Linear error conversion factors for several 
probability levels. 

TWO-DIMENSIONAL ERRORS 

A two-dimensional error is the error in a quantity defined 

by two random variables. For example, consider the position 

of a point referred to x and y axes. Each observation of the 

X and y coordinates may contain the errors v and v . 

X y 
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If the errors are random and independent, each error has 
a probability density distribution of 


^ ^ e. Qfvci p(Vy) —- -—■ €. 

<y%\F2n cj-yvf^ 

"The probability of two events occurring simultaneously 
is equal to the product of their individual probabilities" 
[Ref. 1]. Applying this rule, the two-dimensional probability 
density function becomes: 


p(v,,v,) =_!:_ 

OT^ ct^ 

rearranging terms, 

p(v,,V:>') Q.ix<r»(7v, = ; 



taking the logarithm, 


(-a') jfrvCp iTT cr^ 


Vv 


X 

Vv/ 




For given values of p(v , v ) [physical meaning of 

X y 

p (v , V ) is that the probability that two random variables 
X y 

V and V take values in the interval ±v and ±v ], the left 
X y X y 

side of equation is a constant, k 2 , then 


136 














For several values o£ PC^> ^ family of equal probability 

density ellipses are formed with axes ko^ and kay, A2) . 





Figure A2: Equal probability density ellipses. 

In general, when the two errors are correlated, i.e., 
a change in the one error has some effect upon the other, the 
probability density function, PCv^^, Vy), becomes 
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P (v>(i , ) = 


■■ 


V>cV^ 


"i 

<3^ 


-) 


Xtt 


1 . 


Then, the equation of constant probability density ellipses 
(Figure A3) is 



_i_ 

d-e") 


V» — ‘Ip q- 

<7? 0% (T^y 


where p = correlation coefficient of 


V and v„ 
X y 


and is given by 


^ O-^CTy 

The probability density function integrated over a certain 

region becomes the probability distribution function which 

yields the probability that v and v will occur simultaneously 

X y 

within that region, or: 


P lV^,Vy^ -jj p(v%, Vq Wv^civy 
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Figure A3: Constant probability density ellipse for 
correlated errors. 
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Values of d 


APPENDIX B 

USEFUL GRAPHS FOR THE DETERMINATION OF REPEATABILITY CONTOURS 



Intersection angle, B, in degrees. 
•Figure Al; For ranging systems, the graph of the 

and e/b. 
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Values of e/b 












































rms 



Figure A2: For azimuthal systems, the graph of 
and e/b. 
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Values of e/b 













































LIST OF REFERENCES 


1. ACIC Technical Report No. 96, Principles of Error 
Theory and Cartographic Applications , by Creenwalt, C. R. 
and Shultz, M. E., February 1962. 

2. Bigelow, H, W., Electronic Surveying: Accuracy of 
Electronic Positioning Systems, A.S.C.E. Journal of the 
Surveying and Mapping Division, Volume 89, No. SU3, 

p. 37-76, October 1963. 

3. Bomford, G., Geodesy, Third Edition, The Clarendon Press, 
Oxford, England, 1971. 

4. Bowditch, N., American Practical Navigator, p. 1204- 
1237, Defense Mapping Agency Hydrographic Center, 1977. 

5. Burt, W. A. and others. Mathematical Considerations 
Pertaining to the Accuracy of Position Location and 
Navigation Systems , Fart 1, Naval Warfare Research Center 
Research Memorandum, NWRC-RM34, Stanford Research Institute, 
Menlo Park, California, April 1966. 

6. Hirvonen, R. A. Adjustment by Least Squares in Geodesy 
and Photogramme try , Frederic Ungar Publishing Co., 19^71. 

7. Ingham, A.-E., Hydrography for the Surveyor and Engineer, 
Halsted Press, John Wiley and Sons, 1974, pp. 117-123. 

8. Launlo, S. H., Electronic Surveying and Navigation, 
pp. 85-101, Wiley,~l'97F: 

9. Mikhail, E. M., Observations and Least Squares , lEP, 1976. 

10. Morris R. Heinzen, Hydrographic Surveys: Geodetic Control 
Criteria , M.S. Thesis, Cornell University, December 1977. 

11. Morris R. Heinzen, Hydrographic Surveys: Geodetic Error 
Propagation, paper presented at American Congress on 
Surveying and Mapping, 39th Annual Meeting, March 18-24, 
1979. 

12. P. A. Cross, A Review of the Proposed Global Positioning 
System, The Hydrographic Journal, No. 4, pp. 15-17, 

April 197T: 

13. Schoenrank, R. U., The Determination of Accuracy Lobes for 
Electronic Positioning Systems, Lighthouse pp. 83-92, 

March 1977. 


142 























14. Swanson, E. R., Geometric Dilution of Precision, 

Navigation: Journal of the Institute of Navigation, V. 25, 
No. 4 , pp. 425-429, Winter 19 78-79. 

15. Swanson, E. R., Estimating the Accuracy of Navigation 
Systems, Research Report 1188, San Diego, California, 

U.S. Navy Electronics Laboratory, 24 October 1963. 

16. Veress, S. A., Adjustment by Least Squares, American 
Congress on Surveying and Mapping, Washington, D.C., 1974. 

17. Wolf, P. R., Elements of Photogrammetry, pp. 501-517k 
McGraw-Hill, 1974. 

18. Clair E. Ewing and Michael M. Mitchell, Introduction to 
Geodesy , Elsevier, 1976. 

19. D. B. Thomson and D. E. Wells, Hydrographic Surveying I, 
Lecture Note No. 45, Department of Surveying Engineering, 
Canada. 

20. ACIC Technical Report No. 28, User’s Guide to Understand¬ 
ing Chart and Geodetic Accuracies, by Greenwalt, C. R., 
September 1971. 


143 













11 



INITIAL DISTRIBUTION LIST 


No. Copies 

1. Defense Technical Information Center 2 

Cameron Station 

Alexandria, Virginia 22314 

2. Library, Code 0142 2 

Naval Postgraduate School 

Monterey, California 93940 

3. Department Chairman, Code 68 1 

Department of Oceanography 

Naval Postgraduate School 
Monterey, California 93940 

4. Department Chairman, Code 63 1 

Department of Meteorology 

Naval Potgraduate School 
Monterey, California 93940 

5. LCDR Dudley Leath, Code 68 Lf 20 

Department of Oceanography 

Naval Postgraduate School 
Monterey, California 93940 

6. Professor T. Jayachandran, Code 53 Jy 1 

Department of Mathematics 

Naval Postgraduate School 
Monterey, California 93940 

7. LT Ali Kaplan 2 

Bostanci Koyii 

Gdnen/Balikesir/TURKEY 

8. Deniz Kuve^leri Komutanligi 3 

Personel Egitim Siibe Murdurliigtl 

Ankara/TURKEY 

9. Dz. Kuvvetleri Seyir ve Hidrografi Dairesi Bsk. 3 

Cubuklu, .Istanbul/TURKEY 

10. Deniz Harp Okulu Korn 2 

Heybeliada/Istanbul/TURKEY 

11. Istanbul Teknik Universitesi 2 

Taskisla, Istanbul 

TURKEY 


144 





















12. Orta Dogu Teknik Universitesi 2 

Ankara, TURKEY 

13. Francisco Abreu 1 

Institute Hidrografico 

Rua Das Trinas, 49 

Lisbou-2 

Portugal 


14. John Rees 1 

Defense Mapping Agency HTC 

6500 Brookes Ln. 

Washington, D.C. 20315 

15. Ken Perrin 1 

NOAA Ship Mt. Mitchell 

439 W. Yoak St. 

Norfolk, Virginia 23510 

16. Don Dreves 1 

NOAA Ship Davidson 

FPO 

Seattle, Washington 99798 

17. Peny D. Dunn 1 

CDR/U.S. Naval Oceanographic Office 

NSTL Station 

Bay St. Louis, Mississippi 39522 

18. Luis Faria 1 


Institute Hidrografico 
Rua Das Trinas, 49 
Lisboa - 2 
Portugal 

19. LCDR Douglas A. Backes 1 

Defense Mapping Agency HTC 

6500 Brookes Lane 
Washington, D.C. 20315 

20. LCDR Donald D. Winter 1 

SMC 1745 

Naval Postgraduate School 
Monterey, California 93940 

21. LCDR John Chubb 1 

Chief of Naval Operations, Code Op-952 
Department of the Navy 

Washington, D.C. 20350 


145 






22 . 


1 


Director 

Naval Oceanography Division 
Navy Observatory 

34th and Massachusetts Avenue, NW 
Washington, D.C. 20390 

23. Commander 1 

Naval Oceanography Command 

NSTL Station 

Bay St. Louis, Mississippi 39529 

24. Commanding Officer 1 

Naval Oceanographic Office 

NSTL Station 

Bay St. Louis, Mississippi 39529 

25. Director (Code PPH) 1 

Defense Mapping Agency 

Bldg 56, U.S, Naval Observatory 
Washington, D.C. 20305 

26. Director (Code HO) 1 

Defense Mapping Agency Hydrographic 

Topographic Center 
6500 Brookes Lane 
Washington, D.C. 20315 

27. Director (Code TSD-MC) 1 

Defense Mapping School 

Ft. Belvoir, Virginia 22060 

28. Director 1 

National Ocean Survey (c) 

National Oceanic and Atmospheric 
Administration 
Rockville, Maryland 20852 

29. Chief, Program Planning and Liaison 1 

(NC-2) 

National Oceanic and Atmospheric 
Administration 
Rockville, Maryland 20852 

30. Chief, Marine Surveys and Maps (C3) 1 

National Oceanic and Atmospheric 

Administration 
Rockville, Maryland 20852 


146 











31. 


1 


Director 

Pacific Marine Center - NOAA 
1801 Fairview Avenue East 
Seattle, Washington 98102 

32. Director 

Atlantic Marine Center - NOAA 
439 West York Street 
Norvolk, Virginia 23510 

33. Chief, Ocean Services Division 
National Oceanic and Atmospheric 

Administration 
8060 Thirteenth Street 
Silver Springs, Maryland 20910 

34. Commanding Officer 
Oceanographic Unit One 
USNS BOWDITCH (T-AGS21) 

Fleet Post Office 

New York, New York 09501 

35. Commanding Officer 
Oceanographic Unit Two 
USNS DUTTON (T-AGS22) 

Fleet Post Office 

San Francisco, California 96601 

36. Commanding Officer 
Oceanographic Unit Three 
USNS H. H. HESS (T-AGS38) 

Fleet Post Office 

San Francisco, California 96601 

37. Commanding Officer 
Oceanographic Unit Four 
USNS CHAUVENET (T-AGS29) 

FPO San Francisco, California 96601 


1 


1 


1 


1 


1 


1 


147 


I 










Thesis 

K142A4 

c.l 


I 


B 

190743 


Kaplan . , 

Error analysis or on 

hydrographic position- - 

ing and the applica- es 
tion of least squares. 


IIOVA9 



Thesis 190/ 

K14244 Kaplan 

c.l Error analysis of 

hydrographic position¬ 
ing and the applica¬ 
tion of least squares. 








thesK14244 


Error analysis of hydrographic position! 

1IIIIIII (III nil lllllll 



3 2768 002 11422 5 

DUDLEY KNOX LIBRARY 


















